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Abstract 


^Research  in  computer  'understanding  of  natural  language  has  led  to  the 
construction  of  programs  which  can  handle  a  number  of  different  types  of 
language,  including  questions  about  the  contents  of  data  bases,  stories  and 
news  articles,  dialogues,  and  scene  descriptions.  This  research  draws  on  and 
has  in  turn  had  an  effect  on  many  other  research  areas,  including  software 
engineering,  linguistics,  psychology,  philosophy,  and  knowledge 
representation.  This  paper  provides  a  brief  history  and  overview  of  the 
field,  along  with  ’examples  and  explanations  of  the  operation  of  several 
natural  language  understanding  programs.  The  limitations  of  our'  current 
technology  are  discussed,  and  assessments  are  given  of  the  most  promising 
current  research  directions./ 
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1 •  Introduction 

The  original  purpose  of  this  paper  '/fas  to  give  some  answers  to  the  following 
questions  about  the  state-of-the-art  in  natural  language  understanding  systems: 

What  are  the  limits  now? 

'What  are  the  obstacles  to  progress? 

'.Where  are  the  answers  likely  to  lie? 

In  order  to  be  able  to  answer  these  questions,  I  first  set  out  what  I  feel  are  the 
ma.jor  lines  of  natural  language  research  today,  including  the  study  of  such  topics  as 
knowledge  representation,  metaphor,  "speech  acts"  (the  use  of  language  to  achieve 
goals),  modeling  of  "common  sense"  and  plausibility  .judgement,  relationships  berween 
language  and  perception,  etc.  One  way  to  make  sense  of  this  ootpourri  of  research 
topics  i3  to  consider  the  basic  questions  being  explored  by  two  or  more  of  the 

research  areas.  looked  at  this  way,  I  think  that  the  following  five  questions  are 

motivating  much  of  the  current  research  in  natural  language  understanding: 

(l  )  'What  is  the  function/ purpose  of  language? 

Language  is  in  general  used  by  a  speaker  to  achieve  goals.  Unless  -we  understand 
these  goals,  we  cannot  understand  the  language.  Goals  may  be  extremely  complex:  a 
speaker  may  mean  to  inform,  correct,  or  mislead  a  listener;  or  a  speaker  may  wish  to 
have  the  listener  perform  a  physical  or  cognitive  action,  or  ’undergo  a  certain  kind  of 
experience,  or  answer  questions,  and  so  on.  Often  it  is  necessary  to  'nave  a  model  of 
the  speaker's  ordinary  behavior  in  order  to  understand  the  speaker's  goals  —  the 

language  alone  may  not  be  sufficient.  And  in  order  to  tell  whether  or  not  a  speaker 

i3  telling  the  truth,  a  listener  must  be  able  to  compare  the  speaker's  language  'with 
models  embodying  knowledge  of  human  behavior  as  well  as  the  behavior  of  the  physical 
world.  To  make  matters  even  more  problematic,  any  given  utterance  may  be  'used  to 
serve  quite  different  goals  in  different  situations,  and  a  single  utterance  may  serve 
multiple  goals  simultaneously. 

\2)  'What  does  it  mean  to  "'understand  language"  and 
how  could  we  3how  that  a  system  can  'understand'’ 

Research  attention  has  focused  on  the  sentence  for  a  long  time.  However,  many 
Important  unit3  of  lang’uage  are  much  larger  than  the  sentence:  iial  agues, 
instructions,  scene  and  eve nt  descriptions,  stories,  explanations,  etc.  'We  currently 
lack  the  ability  to  assign  meaning  and  ourpose  to  all  but  the  very  simolest  of  these 


larger  units  of  language. 


(T )  How  carl  a  program  deal  -with  novel  language? 

The  preponderance  of  work  to  date  has  allowed  us  to  deal  -with  novel  syntactic 
structures,  but  we  have  relati.ely  very  little  understanding  of  methods  for  dealing 
■with  novel  semantic  structures,  and  virtually  no  methods  for  dealing  -with  novel 

concepts  expressed  in  language.  He  have  a  need  for  semantic  methods  which  can  give  us 

meanings  for  phrases,  e.g.  "engine  housing  acid  corrosion  damage  report  summary";  we 
also  need  a  dramatically  expanded  'understanding  of  metaphor  and  other  non-literal 
lanugage  (e.g.  "The  soldiers  were  shattered  by  the  experience"  or  "He  found  a 
refrigerator  graveyard.") . 

(4)  How  can  a  program  judge  whether  language  is  meaningful? 

How  do  we  know  that  "The  man  jumped  over  the  fence"  can  be  literally  meaningful 
whereas  "The  cow  jumped  over  the  moon"  oan  not?  How  can  we  decide  that  a  message  is 
garbled  or  that  its  sender  is  deranged?  How  oan  we  decide  that  a  metaphorical 

interpretation  is  intended,  and  how  can  we  know  that  a  given  metaphorical 

interpretation  is  sensible?  To  answer  such  questions  a  system  needs  "common  sense", 
and  canmon  sense  must  3urely  be  based  on  an  extensive  and  detailed  model  of  the 
physical  world,  as  well  as  of  the  worlds  of  human  action  and  inner  experience  re.g. 
perception,  emotion,  memory,  etc.). 

(5)  Hhat  is  the  mo3t  effective  wa y  to  make  the  restricted  natural  language 
systems  of  the  foreseeable  future  seem  natural  to  humans? 

He  have  only  the  beginnings  of  an  ’understanding  of  how  users  -will  behave  -.with 
natural  language  systems.  Thus  there  has  been  to  date  a  fair  degree  of  mismatch 
between  systems  and  users.  He  would  like  to  be  able  to  evaluate  both  existing  systems 
and  future  iesign  alternatives  for  usefulness  ani  convenience.  He  wouli  like  to  able 
give  casual  users  systems  that  allow  natural  expression,  that  do  not  often  surprise 
users  by  not  ’understanding  or  misunderstanding  their  language. 

The  rest  of  thi3  paper  is  organized  historically.  I  could  not  find  a  cood  -/rav  to 
fit  together  the  five  questions  above  to  form  a  coherent  picture  of  the  current 
state-of-the-art  of  research,  and  I  found  it  was  even  more  difficult  to  show  how  the 
current  questions  relatei  to  the  ’ultimate  natural  language  processing  questions.  I 
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discovered,  however,  that  current  research  directions  seemed  much  more  sensible  if 
they  -were  viewed  as  responses  to  specific  shortcomings  of  earlier  'ways  of  looking  at 
the  process  of  natural  language  understanding . 


2.  Ancient  History 

3efore  1940  computers,  if  they  were  thought  about  at  all,  were  considered  to  be 
number  processors.  During  the  40's,  two  major  developments  led  to  the  view  of 
computers  as  somewhat  more  than  simple  number  processors.  The  first  set  of  ideas  was 
due  to  McCullough  and  Pitts,  who  theorized  that  each  neuron  is  a  logical  device 
(roughly  an  .AND  or  DR  gats).  Me  now  know  that  each  neuron  is  far  more  ccmnlex  than 
they  believed  it  to  be,  but  their  ideas  were  important  in  that  they  suggested  that  all 
intelligent  processing,  whether  arithmetic  or  symbolic,  numerical  or  verbal,  could  be 
performed  by  a  single  type  of  mechanism.  Thus  their  views  'were  important  in  a  much 
more  precise  formulation  of  the  brain-computer  analogy  than  had  been  possible  before. 

The  second  major  piece  of  work  -was  Shannon's  -work  on  information  theory;  Shannon 
showed  that  both  numbers  and  text  could  be  treated  as  special  cases  of  a  more  general 
concept  he  called  "information'',  that  information  content  could  be  quantified,  and 
that  ideas  about  information  had  interesting  mathematical  and  practical  applications. 


2. 1  Machine  Translation 

Shannon's  work  led  in  the  early  50's  to  what  I  -will  call  "the  era  of  machine 
translation".  3eing  able  to  treat  text  and  language  in  general  as  information  allowed 
the  possibility  that  language  might  be  manipulated  on  the  new  digital  computers  that 
were  then  being  constructed.  The  initial  idea  for  machine  translation  was  the 
following:  translation  i3  a  process  of  dictionary  look-up,  plus  substitution,  plus 
grammatical  re-ordering.  .As  an  example,  the  3ngli3h  sentence,  "I  must  go  home"  could 
be  translated  into  the  Terman  "Ich  muss  nach  Pause  gehen"  by  substituting  "Tch"  for 
"I",  "muss"  for  "must",  "gehen"  for  "go"  and  "nach  Hause"  for  "home".  In  the  orooess 
two  words,  "nach  Hause"  (to  the  house  1  had  to  be  substituted  for  "home"  —  we  -won't 
worry  here  about  that  fine  point  --  and  a  simple  kind  of  grammatical  re-ordering  hai 
to  take  olace  to  move  the  verb  to  the  end  of  the  sentence. 


Tor  simple  examples  thi3  model  of  the  possibility  of  translation  seems  rather 
intriguing.  However,  it  soon  became  clear  chat  translation  is  really  not  possible 
•without  'understanding .  To  illustrate  the  need  for  'understanding  in  translation,  a 


classic  story  (probably  apochryphal)  describes  the  machine  translation  of  the  phrase 
"The  spirit  is  willing  but  the  flesh  is  weak"  into  Russian  and  then  back  into  ihglish; 
the  translation  is  said  to  have  cane  out:  "The  vodka  is  strong  but  the  meat  is 
rotten." 

Clearly  a  greater  amount  of  world  knowledge  'was  needed;  a  program  had  to 
understand  what  was  being  said  in  order  to  be  able  to  translate  it  properly.  Yet 
another  classic  example  ’was  given  by  Bar-Hillel  in  a  1954  paper  in  -which  he  explained 
-why  he  was  leaving  the  field  of  machine  translation.  Bar-Hillel  cited  the  sentences, 
"The  pen  is  in  the  box"  and  "The  box  is  in  the  pen",  and  pessimistically  stated  that 
he  could  not  imagine  how  a  machine  could  translate  both  sentences  correctly,  assigning 
"pen"  the  meaning  "writing  implement"  in  the  first  sentence,  and  "playpen"  or 
"stockpen"  in  the  second.  While  we  3till  have  a  long  'way  to  go  before  we  could  claim 
to  have  programs  that  truly  ’understand  or  translate  a  significant  range  of  types  of 
language,  we  do  now  know  how  to  write  programs  that  can  appropriately  assign  different 
meanings  to  "pen"  in  Bar-Hillel's  examples  above  by  using  a  system  which  can 
manipulate  simple  spatial  models  of  objects  [ Waltz  30]. 

The  work  on  machine  translation  did  give  a  great  deal  of  imps  Iras  to  work  on 
syntactic  theory  as  evidenced  especially  by  the  work  of  Chomsky  and  also  to  a  degree 
in  the  early  work  on  parsing  high-level  languages  for  compiler  construction,  now  a 
core  topic  in  computer  science. 

To  continue  this  brief  history,  other  major  ideas  that  have  been  influential  in 
the  history  of  natural  language  processing  surfaced  in  the  50's.  I  refer  specifically 
to  the  introduction  of  the  idea  of  heuristic  search  by  'Jewell  and  Simon  in  135o  and 
also  to  the  introduction  of  the  LISP  programming  language  by  McCarthy  in  13^3.  Host 
natural  language  processing  systems  have  been  -written  in  LISP. 

The  entire  field  field  of  machine  translation  essentially  came  to  an  end  in  the 
early  60^3 .  It  is  only  now  ’undergoing  a  kind  of  renaissance,  using  II  models  of 
meaning,  but  the  early  effort  -was  a  nearly  complete  failure. 

3.  The  Semantic  Information  Processing  Sr a 

Cut  of  the  rubble  of  machine  translation  effort  grew  an  effort  that  is  closely 
associated  with  artificial  intelligence.  The  "semantic  information  processing  jra" 

''  roughly  ' 3o2-i  1  produced  a  number  of  ideas  U3ei  in  today's  natural  language 
application  373  tens,  some  of  which  have  proved  to  be  of  practical  value,  "one  notable 
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ideas  of  this  era  are  the  following: 
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(1 )  the  use  of  limited  danains  for  language  understanding  systems;  rather  than 
attempting  to  understand  all  language,  the  limited  domain  approach  i3  to  design  a 
system  that  is  expert  in  one  specific  area  of  language,  but  perhaps  knows  nothing  at 
all  about  any  other  domain; 

(2)  the  "big  switch"  theory  --  to  rationalize  the  study  of  limited  domains  as  a 
contribution  to  a  full  cognitive  theory,  the  "big  switch"  theory  wa3  advanced;  the 
big  switch  theory  holds  that  it  is  possible  to  construct  a  broadly  intelligent  system 
by  generating  experts  in  a  number  of  limited  danains  and  then  piecing  together  a  huge 
system  containing  these  experts  along  'with  a  special  expert,  the  "big  switch",  which 
could  select  the  appropriate  ex^-rt  to  handle  any  given  problem; 

(3)  the  use  of  key  words  to  trigger  certain  actions  —  natural  language  programs 
using  this  idea  look  in  a  sentence  for  one  or  more  key  words  and,  on  the  basis  of  what 
is  found,  take  appropriate  action  (I  give  an  example  below); 

(4)  the  "translation"  of  Siglish  into  formal  languages  —  some  of  the  formal 
languages  that  have  been  used  include  predicate  calculus,  data  base  query  languages, 
and  3ets  of  linear  equations. 

Overall,  we  could  characterize  the  approaches  of  the  50's  to  natural  language 
processing  as  "engineering  approaches",  approaches  which  attempted  to  solve  specific 
problem  danains,  not  to  embody  psychological  reality.  What  do  I  mean  by  "engineering 
approaches"?  Let  us  look  at  some  examples. 

3.1  Keyword  Systems 

The  first  example  i3  the  U3e  of  key  words.  Key  words  were  particularly  Important 
ir.  the  PLICA  and  DOCTOR  programs  written  by  Weizenbaum  and  the  PARRY  program 
f which  simulated  a  paranoid  person)  by  Colby  and  hi3  collaborators 


PATTERN 


(*  computers  *) 

[*  mother  *'> 

< no thing  matches? 


RESPONSE 

Do  canputers  frighten  you? 

Tell  me  more  about  your  family. 
Please  go  on. 


Figure  1.  Simplified  ELIZA  patterns  and  responses. 


In  Figure  1  (a  highly  simplified  example  based  on  ELIZA)  matches  any  word  or 
list  of  words  (including  no  words  at  all)  and  the  literal  -words  such  as  "computers" 
can  only  match  words  like  "computers".  Thus  if  someone  -were  to  type  "I  hate 
canputers"  to  the  ELIZA  program,  it  might  respond,  "Do  computers  frighten  you?"  If  the 
person  typed,  "Jfy  mother  is  an  electrician,"  ELIZA  could  respond,  "Tell  me  more  about 
your  family".  ELIZA  was  also  capable  of  using  phrases  and  -words  which  matched 
patterns  to  construct  responses;  thus,  it  could  respond  to  "I  believe  that  <x>"  with 
"How  long  have  you  believed  that  <x>". 

3. 2  Translating  English  into  £  Fornal  System 

As  an  example  of  the  translation  of  English  into  a  formal  language,  consiier 
Bobrov's  STUDENT  program  [Bobrow  1 96b]  which  translated  algebra  -word  problems  into  a 
set  of  linear  equations.  STUDENT  treated  each  input  sentence  as  though  it 
corresponded  to  a  simple  equation;  thus,  "John's  age  now  i3  two  times  Mary's  aze” 
would  be  translated  into  an  equation  such  as  "JA  =  2  *  NA".  In  order  to  perform  this 
translation,  Bobrov's  program  had  to  note  that  John's  age  now  is  a  variable  'JA\ 
Mary's  age  i3  a  variable  (MA),  and  "is  two  times"  should  be  translated  info  "2  *"  in 

the  equation.  Similarly  the  equation,  "In  three  .years  John  will  be  six  years  older 
than  Mary"  translates  into  the  equation  "JA  +  3  “  MA  +  5".  This  program,  once  it  had 
formed  as  many  equations  as  variables,  could  then  pass  the  equations  to  another 
program  that  was  expert  at  solving  simultaneous  linear  equations.  The  idea  of 
translating  English  into  formal  languages  has  led  to  many  other  programs  including 
nos o  of  the  current  generation  natural  lang-iage  data  base  "front  ends". 

'.3  Data  Base  Question- Answering 

Another  precursor  of  data  base  query  generation  frcm  English  -.was  the  3A.FE3A1L 
program  of  IreenT  1  BAoEBALL  had  a  tabular  data  base  much  like  that  shown  in 

Fig-ore  2a,  containing  information  about  all  the  games  played  in  the  American  Leag-ie 
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during  one  season.  When  given  a  question  3uch  as  "Who  did  the  Yankees  play  on  July 
“’7",  the  3A3E3ALL  program  turned  this  into  a  query  template  similar  to  the  one  3hown 
in  Figure  2b.  3ASSBALL  couli  then  compare  this  query  template  with  the  iata  base  and 
return  the  answer  "Red  Sox". 


MONTH 

PLACE 

IAY 

iame 

'VINNER /SCORE 

LOSER/ SCORE 

July 

Cle'/eland 

6 

95 

White  Sox/  2 

Indians/  0 

July 

Boston 

n 

i 

96 

Red  Sox/  5 

Yankees/  t 

-July 

Detroit 

1 

97 

Tigers/  10 

Athletics/  2 

Figure  2a.  BASEBALL'S  data  base. 

(JR  (July  7  —  Yankees/  —  ° ANSWER/  — ) 

(July  7  —  7ANSWER/  —  Yankees/  —)) 

Figure  2b.  A  query  template  in  BASEBALL. 


.ALL  these  crograms  illustrate  some  of  the  kinds  of  "engineering  techniques"  that 
were  used  to  handle  language  during  the  60s,  techniques  which  illustrated  the 
simplifications  possible  through  the  restriction  of  inputs  to  narrow  semantic  domains, 
and  'Which  offerred  the  promise  of  near-teim  practical  applications.  ’Jn fortunately, 
the  programs  developed  using  these  techniques  shed  very  little  light  on  the  cognitive 
processes  -underlying  language  can  prehension. 

i .  ITT.  The  Flowering  of  .Semantic  Information  Processing  and  Seeds  of  Cognitive 
Science 

The  years  around  1?70  proved  to  be  noteworthy  for  a  number  of  reasons.  I  will 
describe  briefly  several  well-known  and  influential  programs  that  appeared  around 
*9'70,  and  -.which  pushed  the  notion  of  semantic  information  processing  to  its  ultimate 
limits. 


-l.l  3HRDLU 


The  first  program  is  Winograi's  .712717 
assumed  that  two  main  analogies  'were  true 


analogous  to  programs,  that 


sentences 


‘  Wi  nog  rad 
.  Tne 
couli  be 


’  T"2 ..  Wi nograd '3  program 
first  '.was  that  .sentence  ver* 
"understood"  bv  transforming 
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them  into  programs.  The  programs  thus  created  could  then  be  used  to  carry  out  various 
tasks  (e.g.  moving  blocks  on  a  table),  or  search  for  information  in  SHRDLU's  data 
base,  or  generate  an  answer  for  its  user.  The  second,  related  analogy  was  that  words 
correspond  to  program  3teps.  Thus,  word  "definitions"  for  SHRDL'J  were  program 
fragments  in  the  MICROPIANNER  programming  language  ^Sussma.n  and  McDermott 
MIDROPLANNER,  inspired  by  Hewitt's  [l969]  PLANNER  language,  -was  centered  around  the 
ideas  of  heuristic  search.  Thus,  ’Vinograd's  program  unified  the  much  earlier 
heuristic  search  material  with  natural  language  processing.  SHRDLU's  grammar  itself 
operated  by  a  heuristic  search  process;  it  tried  out  a  certain  interpretation  of  a 
sentence,  and  if  it  could  not  make  syntactic  or  semantic  sense  of  the  sentence,  SHRDL'J 
would  back  up  and  try  a  different  interpretation. 

SHRDL'J  solved  a  broad  set  of  problems,  and  ’was  able  to  handle  sentences 

exhibiting  a  wide  variety  of  linguistic  phenomena.  It  interpreted  declarative 

sentences  as  data  base  updates,  interrogative  sentences  as  data  base  searches,  and 
Imperative  sentences  as  specifications  for  goals;  these  goals  were  achieved  by  first 
forming  and  then  executing  a  plan,  which  generally  involved  data  base  search  and 
update  as  well  as  sentence  generation.  Vinograd's  dissertation  also  included  detailed 
analyses  of  SHRDLU's  limitations,  and  listed  a  number  of  examples  that  -would  be 
difficult  to  handle  -within  Vinograd's  paradigm.  SHRDLU's  linguistic  coverage  -was  very 
broad  compared  to  previous  programs.  It  was  able  to  handle,  for  instance, 
quantification,  some  kinds  of  pronouns,  and  negation.  It  -was  able  to  learn  word 

definitions,  generate  natural-sounding  dialogue,  and  answer  questions  about  the 

history  of  its  dialogue  and  plan  execution.  .An  example  of  the  operation  of  Vinograd's 
program  is  3hown  in  Figure  M. 
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Jbtda.  block  t+itic/l  is  bzller  #tax  Hie  one 
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The  second  piece  of  work  fran  around  1970  which  I  would  like  to  discuss  is  Woods' 
LUNAR  program  _ Woods  et  al  19'72],  which  was  a  natural  lang’uage  front  end  for  a  data 
base  containing  moon  rock  sample  analyses.  7or  parsing  sentences  (i.e.  finding  the 
syntactic  structure  of  the  sentences)  Woods  used  Augmented  Transition  Networks  ( ATNs ) 
'Woods  1970],  which  implemented  a  heuristic  search  much  like  the  kind  that  Winograd 
used  in  3HRDLU.  Woods'  formulation  -was  so  clean  and  natural  that  it  has  been  used 
since  then  for  most  parsing  and  language  'understanding  systems.  Woods  also  introduced 
a  very  general  notion  of  quantification  based  on  predicate  calculus  and  used 
sophisticated  techniques  to  translate  questions  into  data  base  queries.  An  example  of 
a  sentence  that  Woods'  LUNAR  program  could  answer  is:  "live  me  all  analyses  of 
samples  containing  olivine." 

Both  LUNAR  and  3HRDLU  were  comprehensive  systems;  both  could  use  relatively 
unconstrained  language;  both  worked  in  very  narrow  domains,  but  had  complete, 
privileged  knowledge  of  their  worlds.  (LUNAR  knew  everything  that  could  be  known 
about  the  data  base  of  lunar  rocks;  3HKDLU  was  the  keeper  of  the  block's  -world.)  Both 
also  have  proved  to  be  non- portable  and  non-extensible.  Although  there  -were  several 
attempts,  no  serious  production  programs  ever  developed  from  either  of  the  pieces  of 
work.  Both  -were  prototypes  which  had  a  limited  life  and  are  now  no  longer  used. 

1.3  NLP3 

NLPQ,  a  third  interesting  program  frcra  around  1970,  came  out  of  the  -work  of 
leorge  Re  id  o  m  on  the  use  of  natural  language  to  set  up  simulations  ^Heiiorn  1?"i '. 
For  example,  given  the  following  sentences  (a  partial  transcript  of  the  program's 
operation)  Heidom's  program  could  set  up  a  simulation,  and  run  it  to  answer 
questions : 

User:  When  a  vehicle  arrives  at  a  station,  it  leaves  there  immediately  if  the 

length  of  the  line  at  a  pump  in  the  station  is  not  less  than  2. 

--  oercsnt  0f  the  vehicles  are  cars  and  a  fourth  3re  trucks. 

Fnere  i3  .just  one  pump. 

A  simulation  time  of  3  hours  is  desired. 

A3k  questions  for  further  info. 

System:  ROW  OFTEN  00  THE  7EHI0LES  ARRIVE  AT  THE  STATION0 

User:  The  arrivals  of  vehicles  are  normally  distribute!  with  a  mean  of  9 

minutes . 


?- ige  1 2 


<3ystera  asks  more  questions,  eventually  judges  that  the  problem  statement  is 
complete,  and  can  then  answer  questions  about  the  situation  that  was  described 
to  it  or  about  the  simulation  itself> 

Heidom's  program  embodied  a  model  of  what  a  complete  simulation  would  have  to  include 
and  was  able  to  ask  questions  of  the  user  if  the  information  that  was  given  was 
insufficient.  Thus,  Heidom's  program  embodied  a  kind  of  world  knowledge  about  what 
constitutes  a  complete  formulation  of  a  problem. 

1.4  MARS 15 

Also  around  1970  another  influential  piece  of  work  by  Roger  Schank  -was  completed. 
This  -work  has  continued  to  this  day.  Schank  has  dealt  -with  much  more  unconstrained 
language,  particularly  language  about  human  actions.  Schank' s  work  was  based  on  the 
development  of  a  set  of  "primitives  of  conceptual  dependency".  .411  sentences  input  to 
Schank's  systems  are  translated  into  structures  centered  around  a  small  number  of 
primitives.  The  primitives  (which  have  changed  a  little  over  the  years,  and  have 
varied  in  number  from  14  to  16)  include  MTRANS,  ’which  stands  for  "transfer  of  mental 
information";  ATRANS,  -which  stands  for  "transfer  of  possession";  PTRANS,  -which 
stands  for  "physical  transfer  of  an  object  from  one  location  to  another";  CONC,  short 
for  "conceptualize",  or  think  about;  M3UILD,  -which  stands  for  "build  memory 

structures";  ATTEND,  which  covers  see,  hear,  taste,  smell,  touch;  RR0P5L,  -which 

stands  for  "the  application  of  physical  force  to  an  object";  MOVE,  that  is,  move  a 
body  part;  GRASP,  that  is  hold  in  one's  hand;  INGEST,  and  EXPEL.  Sentence  meaning 
representations  are  formed  by  using  these  primitives  in  conjunction  with  other  words 
in  a  sentence  to  form  a  kind  of  "semantic  network"  (see  examples  below') .  Each 

primitive  of  conceptual  dependency  is  also  associated  with  a  case  grammar- like  frame 
jEillmore  1963]  that  specifies  which  words  can  occur  with  the  primitive  in  a  sensible 
manner.  Thus,  for  instance,  MTRANS  (the  transfer  of  mental  information)  requires  that 
there  be  an  intelligent  source  for  the  mental  information  and  an  intelligent  recipient 
for  the  information.  (>TTRANS  i3  the  primitive  used  internally  to  represent  such 

iiverse  words  as  tall,  hear,  say,  speak,  read,  etc.) 

lonceptual  dependency  primitives  have  been  used  not  only  to  represent  meaning  but 
also  to  organize  expectations;  for  example,  having  decided  that  mental  information 
•.was  transferred  to  a  hearer  Schank's  programs  could  predict  that  the  hearer  would 
thereafter  have  that  information  available  in  memory. 


Page  15 


The  MAHGIS  program  .Schank  et  al  1973]  could  accept  3imple  sentences  and  answer 
questions  about  them,  generate  paraphrases  of  those  questions,  and  make  inferences 
based  on  the  questions.  For  example,  given  the  statement,  "John  gave  Mary  an 
aspirin,"  the  inference  program  generated  sentences  such  as:  "Mary  felt  sick,"  "Mary 
wanted  to  feel  better,"  "John  Wanted  Mary  to  feel  better,"  "Mary  asked  John  for  an 
aspirin,"  etc.  (All  these  were  viewed  as  plausible,  not  necessary  inferences.) 

Figure  4-  shows  some  examples  of  conceptual  dependency  diagrams  corresponding  to 
input  sentences.  In  Figure  4a  is  a  structure  corresponding  to  "John  grew  six  inches". 
One  can  read  this  roughly  as  "John's  size  went  from  some  value  X  to  some  value  X  +  6 
inches".  The  representation  of  the  apparently  similar  sentence,  "John  grew  com"  is 
quite  different,  as  shown  in  Figure  4b.  Thi3  structure  can  be  roughly  read,  "John  did 
something  (unspecified)  which  caused  the  3ize  of  the  com  to  go  from  some  size  X  to 
some  size  X  +  Delta". 

Figure  4c  is  a  conceptual  dependency  diagram  corresponding  to  a  sentence,  "John 
gave  Mary  a  bicycle."  This  structure  can  be  roughly  read ,  "John  transferred  possession 
of  the  bicycle  from  himself  to  Mary." 

In  Figure  4d  the  related  sentence,  "Mary  got  a  bicycle  from  John"  has  a  very 

similar  representation  except  that  Mary  is  listed  as  the  agent,  i.e.  the  actor  who 

caused  the  transfer  of  the  bicycle  from  John  to  her. 

4.5  Other  Ideas  from  the  early  70's 

AI30  in  the  early  70's  there  were  several  other  contributions  that  have  played  an 
Important  role  in  defining  current  research  topics.  Two  such  contributions  were  made 
by  3earle  0)  and  Grice  ']  1 975 ]  on  "speech  act  theory". 

Speech  act  theory  attempts  to  account  for  the  purposes  for  ’which  language  is 
used,  as  opposed  to  the  (logical)  meaning  of  individual  sentences.  .As  an  example,  if 
given  the  sentence,  "Gould  you  pass  the  salt,"  we  ’understand  this  not  as  a  request  for 
information  about  whether  -we  are  physically  capable  of  passing  the  salt,  but  as  a 

request  to  carry  out  the  action  of  actually  doing  so.  In  this  sense,  speech  act 

theory  pints  out  that  sentences  are  not  analogous  to  programs  —  that  is,  that  no 
direct  translation  of  a  sentence  into  a  program  form  will  capture  all  its  meaning1’  3s' . 
Language  consists  of  act3  by  speakers,  and  as  3uch,  the  intentions,  goals,  strategies, 
and  beliefs  of  both  speakers  and  listeners  are  of  central  importance  in  -understanding 
language. 
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5.  Lessons  of  the  ’’O's 

Other  phenomena  we re  noted  during  the  ''O's  that  we  3till  do  not  know  how  to  deal 
■with  -well.  These  include  processing  language  which  falls  outside  of  a  narrow  domain, 
handling  dialogue  (which  can  itself  be  the  topic  of  conversation  in  any  dialogue) , 
insuring  rapid  response,  and  meeting  other  "human  factors"  requirements  (such  as 
presenting  information  to  a  user  of  the  natural  language  system  in  a  ’/ray  that  is 
’unambiguous,  teaches  the  user  about  the  system's  abilities  and  limitations,  etc.). 
Speed  is  especially  important.  If  users  have  to  wait  a  long  time  for  a  response,  they 
can  became  very  impatient,  so  efficiency  of  the  algorithms  for  natural  language 
processing  is  of  importance,  (i'll  have  more  to  say  about  3ome  of  these  issues  on 
dealing  ’with  real  users  when  I  talk  about  the  accomplishments  of  the  PLANES  system.) 

5. 1  Knowledge  Representation 

Another  major  realization  during  the  70's  has  been  that  knowledge  representation 
formalisms  are  of  central  importance  to  all  natural  language  processing.  3efore  we 
can  put  knowledge  into  a  system,  we  need  to  be  able  to  represent  that  ’Knowledge 

appropriately,  in  a  manner  which  allows  the  'Knowledge  to  be  found  and  used  -when 
appropriate  during  the  natural  language  understanding  process.  This  need  has  been 
pointed  out  for  a  long  time  by  John  McCarthy  [1968]  and  is  now  generally  recognized  as 
the  central  issue  in  artificial  intelligence.  Among  the  issues  in  knowledge 

representation  are:  how  should  items  in  memory  should  be  indexed  and  accessed,  how 
should  context  be  represented,  how  should  memory  be  updated,  how  can  programs  deal 
with  inconsistency  —  that  is,  if  we  have  new  information  to  be  added  to  our  knowledge 
base  which  is  inconsistent  with  the  information  currently  there,  how  should  we  store 

the  new  information?  can  we  (and  should  we)  resolve  the  conflict?  If  not,  -which 

information  should  we  act  on,  or  should  to  somehow  integrate  both  parts  of  the 
conflicting  information  into  our  action?  Various  -ways  have  been  suggested  for 
handling  this  sort  of  conflict,  for  example,  partitioning  memory  into  a  number  of 
possible  "contexts",  each  of  -which  is  internally  consistent.  .Another  important 
problem  i3  that  of  deciding  -whether  and  'now  -we  could  know  that  a  ’.Knowledge 
representation  scheme  i3  sufficient  and  complete,  so  that  to  could  be  assured  that  any 
kind  of  knowledge  imaginable  could  be  represented  in  the  scheme. 
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5.2  Common  Sense 


We  came  to  realize  during  the  70's  that  we  needed  to  endow  natural  language 
programs  with  "ccmmon  sense",  which  can  only  be  based  upon  a  body  of  knowledge  of  the 
outside  -world.  In  understanding  language,  people  bring  a  large  amount  of  information 
to  bear  which  cannot  be  deduced  from  the  language  itself.  A  sentence  is  never  a 
formula  or  a  program  -Aich  is  complete  in  and  of  itself.  No  process  of  rewriting  a 
sentence  could  be  sufficient  to  construct  all  the  meanings  that  a  hearer  get3  from 
listening  to  the  sentence.  In  many  ways,  language  is  a  kind  of  shorthand  or  set  of 
index  items;  a  listener  uses  these  as  keys,  and  retrieves  from  memory  the  rest  of 
(appropriate)  information  that  must  be  added  to  the  language  in  order  to  formulate  its 
full  meaning. 

Consider,  for  example,  the  following  sentences  (from  [Vinograd  1972]): 

The  city  councilmen  refused  to  give  the  women  a  permit  ot  march  because 

(a)  they  feared  violence. 

(b)  they  advocated  revolution. 

In  (a)  they  seems  to  refer  to  the  city  councilmen,  whereas  in  (b)  they  refers  to  the 
women.  How  do  -we  judge  this?  The  structures  of  the  two  sentences  are  identical,  so 
they  cannot  help  us  to  distinguish  the  two  cases.  The  only  answer  seems  to  be  that  we 
know  a  great  deal  about  human  behavior,  and  can  readily  access  and  apply  this 
’Knowledge  -when  it  is  needed  in  understanding  language. 

One  problem  requiring  ccmmon  sense  is  the  problem  of  .judging  whether  or  not  a 
sentence  is  even  meaningful.  Related  problems  involve  choosing  the  most  appropriate 
reading  -/*ien  several  are  possible,  and  making  appropriate  inferences  about  sentences 
or  text  passages.  Some  research  from  the  mid-70's  gave  some  tentative  answers  to 
these  kinds  of  problems. 

5-7  Frames 

In  his  "frames  theory"  Minsky  ^975]  suggested  that  we  needed  to  be  able  to  ieal 
•with  much  larger  memory  units  than  had  been  considered  before.  He  offerred  as 
candidates  for  the  memory  unit3  "frames",  strictures  consisting  of  a  core  and  slots: 
each  3lot  corresponding  to  either  a  facet  or  participant  of  a  concept  embodied  in  the 
frame,  or  a  3pace  for  a  pointer  to  a  related  concept  e.g.  an  instance  of  the  frame's 
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concept  or  a  variation  on  the  frame).  Minsky  argued  that  an  important  function  of 
frames  ras  to  represent  stereotypes;  stereotypes  provide  a  neat  explanation  for 
"default  reasoning",  the  process  by  which  we  take  the  shorthand  information  available 
in  language,  and  retrieve  and  fill  in  the  rest  of  the  information  that  would 
ordinarily  be  be  expected  in  that  situation. 

Frames  were  also  suggested  for  modeling  context;  that  is,  a  context  could  be 

represented  as  a  frame  which  in  turn  -would  contain  as  slot  values  other  frames  that 

ought  to  be  present  in  that  context. 

5.4-  SCRIPTS 

Another  larger  processing  unit  specialized  for  stories  is  the  SCRIPT,  proposed 
and  developed  by  Roger  Schank  and  his  collaborators  at  Yale  [ochank  and  Abelson  IThI. 
SCRIPTS  correspond  to  stereotypes  for  stories,  and  are  proposed  as  the  kind  of 
information  that  allow  us  as  listeners  of  a  story,  to  fill  in  unmentioned  details  and 
make  appropriate  inferences.  SCRIPTS  also  can  also  provide  a  plausible  mechanism  for 
expectation-driven  text  analysis.  If  we  know  a  story  is  about  a  restaurant,  we  expect 
that  we  may  encounter  a  waitress,  menu,  table,  a  bill,  food,  and  other  specific  'Kinds 
of  information;  SCRIPTS  provide  a  kind  of  ready-made  framework  for  encoding  that  kind 

of  information.  As  an  example  of  the  use  of  a  script,  consider  the  following  story  (a 

simplified  version  of  a  story  actually  used  by  Wendy  Lehnert  [l977]). 

John  took  the  bus  from  Mew  Haven  to  New  York.  On  the  way,  his  pocket  -was 
picked.  He  went  to  Hama  Leone's  and  ordered  spaghetti.  John  couldn't  pay  the 
bill,  so  he  washed  dishes. 

What  did  John  eat? 

Notice  that  in  the  passage  it  -.was  never  mentioned  that  John  ate  spaghetti.  It 
3ays  that  he  ordered  spaghetti,  yet  we  make  the  inference  that  he  actually  ate  the 
spaghetti  in  the  absence  of  any  information  to  the  contrary.  Lehnert's  program  used  a 
restaurant  SCRIPT  to  make  plausible  inferences. 

5.5  Non- literal  Lang age 

Another  realization  of  the  70's  -was  that  -we  needed  different  or  new  techniques 
for  dealing  with  non-literal  language.  .As  pointed  out  by  a  number  of  workers, 
metaphor  i3  a  pervasive  phenomenon  in  language .  Typically  words  have  many  senses 
which  are  not  neatly  captured  by  a  simple  definition.  Words  can  be  applied  in  rove 1 
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situations  which  are  difficult  to  predict  from  any  number  of  dictionary  definitions, 
dome  attempts  to  deal  with  nonliteral  language  included  the  "preference  semantics"  of 
Wilks  [l976j,  and  3ecker's  [1975]  "phrasal  lexicon",  a  compendium  of  idioms  'which 
cannot  be  understood  as  a  composition  of  simple  'word  definitions.  Examples  include 
"big  as  a  bam",  "sly  as  a  fox",  "dry  as  a  bone",  and  so  on. 

It  was  also  recognized  that  new  techniques  were  needed  for  dealing  'with  language 
units  that  were  larger  than  sentences.  3uch  instances  included  stories  or  news 
articles,  dialogues,  and  descriptions  or  instructions. 

5.6  Evaluation  and  Data  on  Users 

One  of  the  difficulties  in  evaluating  the  current  state  of  the  art  in  natural 
language  processing  is  that  most  papers  only  give  positive  examples  of  the  operational 
systems.  There  is  no  way  to  tell  viiether  these  positive  examples  are  typical  of  the 
operation  of  the  system,  or  whether  they  are  an  exhaustive  list  of  all  the  questions 
the  system  ha3  ever  answered  appropriately.  In  addition,  we  have  little  information 
on  how  users  will  behave  -with  a  natural  language  system  if  they  are  not  constrained. 
We  do  have  some  experience  'with  tests  where  a  human  simulates  a  computer  [Malhotra 
1975][Tennant  19B0].  In  such  tests  users  sit  at  a  terminal  and  type  into  it  as  though 
they  -were  typing  to  a  natural  language  understanding  program,  -when  in  fact  they  are 
typing  to  another  terminal  where  a  person  i3  sitting  pretending  to  be  a  natural 
language  'understanding  program.  Such  tests  are  probably  too  'neons trained  because  the 
person  simulating  the  natural  language  system  is  obviously  capable  of  'understanding 
all  sort3  of  language.  Uonetheless,  such  tests  are  very  instructive  in  gaining  an 
understanding  of  how  users  would  behave  with  an  'ultimate  natural  language  3vstem. 
Such  tests  give  much  less  information  about  what  the  minimum  features  should  be 
included  '.n  order  to  make  a  useful  natural  language  system.  Some  of  the  questions  we 
would  like  to  know  about  user  behavior  are  the  following:  Which  features  should  a 
system  have?  Which  must  it  have?  Which  features  are  the  most  Important?  What 
conputational  model  most  naturally  handle  such  features?  And  finally,  is  it  possible 
to  have  a  restricted  natural  language  system  that  would  be  truly  convenient  for  a 


casual  user0 
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6.  Natural  Language  ?ront  Ends  for  Data  3a3es 

During  the  7Q's  a  number  of  natural  language  lata  base  front  ends  appeared: 
LUNAR  [Woods  et  al  1972]  has  already  been  briefly  described;  other  systems  included 
REL  [Thompson  et  al  1969,  1975J,  an  English-like  extensible  system;  LIPER/LADDER 
[Hendrix  et  al  1973];  REQUEST  [Plath  1976];  and  ROBOT  ['Harris  197^]. 

6.1  PUNES 

As  an  example  of  the  engineering  approach  and  to  give  a  more  complete  idea  of  the 
current  state  of  the  art  of  natural  language  processing,  I  would  like  in  this  section 
to  discuss  the  PLANES  system  developed  at  the  University  of  Illinois.  PLANES  is  a 
natural  language  data  base  front  end  that  works  on  a  large  relational  data  base  of 
aircraft  flight  and  maintenance  data.  PLANES  assumes  that  all  the  language  it  obtains 
is  in  the  form  of  requests  which  it  turns  into  formal  query  language  expressions.  It 
then  runs  the  query  language  expressions  on  the  data  base  and  returns  an  answer  to  a 
user  in  an  English- like  or  tabular  form.  PLANES  uses  a  "semantic  grammar",  that  is. 
it  has  ATN  parsers  for  every  kind  of  phrase  that  can  occur  in  its  world:  time 

phrases,  phrases  referring  to  aircraft,  places,  etc.  The  goal  of  the  parsing  phase  of 
PLANES  is  a  set  of  semantic  constituents;  so  for  example,  the  sentence,  "Which  planes 
had  ID  or  more  flights  during  January  1970?”  yields  the  semantic  constituents  "Which 
plane";  "greater  than  ten  flights";  and  "January  1 970" .  The  goal  of  the  front  end 
is  to  take  these  constituents  and  fit  them  into  a  "query  template".  The  query 
template  may  not  be  filled  in  completely  by  the  information  that  was  given  in  the 
English  sentence  and  in  this  case,  PLANES  looks  back  through  the  dialogue  to  locate 
and  fill  in  missing  items.  Thus,  given  the  pair  of  requests: 

Which  aircraft  required  more  than  10  hours  maintenance  in  June  IT7?? 

< answers  question> 

July? 

PLANES  would  use  the  information  from  the  first  sentence  in  formulating  a  query 
expression  for  the  second  sentence. 

PLANES  was  designed  from  an  engineering  point  of  view  and  makes  no  pretense  of 
modeling  psychological  reality.  Some  of  the  advantages  of  the  design  of  planes  are 
the  following:  (l  )  It  allows  a  nongrammatical  input.  The  phrases  can  occur  in  any 
order  30  that,  for  instance,  the  sentence,  "\7a  January  1 unscheduled  maintenance" 
is  a  reasonable  request  for  PLANES.  [2 )  PLANES  can  handle  ellipsis,  as  illustated  in 
the  example  above;  [3'  PLANES  can  also  handle  some  forms  of  oronoun  reference  in  a 
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manner  similar  to  the  way  it  handles  ellipsis.  (4'  It  can  deal  -with  unforeseen 
requests  because  it  is  able  to  use  the  query  template  to  do  expectation-driven 
analysis  of  3uch  requests.  It  fills  in  the  query  template  by  categorizing  all  the 
constituents  it  finds  and  then  inserting  those  constituents  in  the  appropriate  slots 
in  the  query  template. 

PIANES  handles  speech  acts  by  matching  all  the  speech  act  words  it  can  find  and 
then  ignoring  them;  PIANES  always  assumes  that  the  user  is  requesting  information. 
Thus  PIANES  matches  and  then  throws  away  all  such  portions  of  requests  as  "Can  I  get 
...",  "Could  I  get  ...",  "Can  I  have  ...",  "Could  you  give  me  ...  ",  "Could  you  show 

...",  "Could  you  get  ..."  and  so  on. 

PIANES  .judges  the  plausibility  or  meaning  fulness  of  questions  through  reference 
to  what  we  call  "concept  case  frames".  Some  examples  of  concept  case  frames  are 
[<planeXreceiveXmaintenance>  ]  or  [< pi aneXflyX flight  hours>l.  The  former  can  match 
sentences  3uch  as  "Which  planes  had  maintenance  in  January?"  or  "Did  any  planes  have 
unscheduled  maintenance  during  January?"  and  the  latter  can  match  sentences  such  as 
"How  many  flight  hours  did  plane  7  log  in  February?"  Concept  case  frames  are  also  used 
for  ellipsis  and  pronoun  reference.  If  some  constituents  are  missing  PIANES  uses 
concept  case  frames  to  decide  which  constituents  it  needs  to  locate  in  the  preceding 
dialogue  in  order  to  form  a  complete  request. 

Some  aspects  of  plausibility  judgement  are  also  embodied  in  the  semantic  grammar 
since  PIANES  parses  time  phrases,  place  phrases,  airplane  phrases,  and  maintenance 
type  phrases  separately.  It  can  make  judgements  as  to  'whether  each  of  these 
individual  phrases  is  meaningful. 

6.2  'Jser  Evaluation  of  PIANES 

In  testing,  we  have  found  that  users  often  ask  vague  and  complex  questions.  For 
example,  users  ask  for  reports  3uch  a3  in  the  following:  "Jive  me  a  month  by  month 
status  report  for  ?-i's."  The  system  is  simply  incapable  of  deciding  easily  what 
"status  report"  means  in  this  example.  In  other  cases,  the  system  may  be  asked  to 
make  .judgements  as  in  the  sentence,  "Which  plane  had  the  -.worst  maintenance  record?" 
The  problem  here  i3  that  -worst  maintenance  record  ean  mean  different  things  to 
different  users.  Is  the  worst  plane  the  one  that  had  the  most  hours  out  of  service, 
or  is  it  the  one  that  cost  the  most  to  repair,  or  is  it  the  plane  that  required  the 
most  maintenance  hours  during  the  month,  or  i3  it  the  cia33  of  plane  that  crashed  most 


often  that  month?  Without  'Knowing  more  about  the  user  and  'ni3  interests,  we  simply 
cannot  make  such  a  lodgement;  as  it  stands,  PLANES  cannot  even  be  programmed  to 
include  representations  of  these  various  possibilities. 

Users  often  input  declarative  information.  For  example  users  may  say  to  she 
system,  "I'm  only  interested  in  A-7's".  While  it  can  leal  with  certain  special  cases 
of  this  sort,  PLANES  is  incapable  of  dealing  with  such  sentences  in  an  appropriate  -way 
in  general;  it  assumes  that  user  inputs  are  requests,  unless  it  recognizes  them 
specifically. 

Users  often  refer  to  itans  ’which  are  not  in  the  data  base.  For  example,  a  user 
can  refer  to  concepts  frcm  earlier  sentences  or  answers  to  previous  questions,  and 
PLANES  is  not  currently  capable  of  dealing  -with  such  questions.  In  addition,  users 
sometimes  refer  to  items  that  are  not  covered  -within  the  data  base  scope.  For 
instance,  our  data  base  has  no  information  on  pilots  or  the  sources  and  destinations 
for  specific  flights.  Yet  a  user  might  reasonably  be  interested  in  asking  such 
information  and  PLANES  would  simply  say,  "I  did  not  understand  your  request." 

Hewriting  PLANES  for  a  new  data  base  would  be  difficult.  Much  of  the  information 
in  PLANES,  including  the  semantic  grammar  fbr  handling  plane  phrases  a nd  time  phrases, 
would  not  carry  over  to  a  different  world,  and  a  new  semantic  grammar  -would  have  to  be 
generated . 

PLANES  does  not  respond  well  -when  it  only  partially  understands  the  question. 
Sometimes  in  such  oases  PLANES  -will  3imply  say,  "I  did  not  understand  your  question." 
If  no  answer  i3  found,  PLANES  3impl,y  says,  "I  found  no  answer."  It  could  be  that  no 
answer  was  found  because  there  wa3  no  data  of  the  sort  the  user  -was  looking  for,  and 
PIA'IES  -would  not  easily  allow  a  user  to  distinguish  this  case  from  the  one  where  there 
were  no  instances  of  the  specific  type  of  event  the  user  wa3  looking  for.  This  latter 
ability  is  being  added  to  PLANES. 

6. M  Evaluation  with  Tasual  Users 

In  recent  testing  ^Tennant  19*30],  users  were  briefly  (in  less  than  1"  minutes) 
introduced  to  PLANES.  Users  were  chosen  who  -were  already  familiar  with  aviation, 
flight,  and  maintenance  operations.  >ut  of  a  total  of  402  queries  they  enters; ,  T~ 
were  understood  correctly  by  PLANES.  Ten  were  rejected  by  PLANES  as  unintelligible 
and  l’7  produced  errors;  that  i3,  PIA'IES  lid  something  incorrect.  If  the  11”  errors, 
were  iue  to  3imple  emissions  in  the  iictionary,  grammar  cr  query  generator  ani  so 
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could  be  easily  fixed,  while  the  other  10  failed  because  of  inadequacies  in  the 
formalisms  of  PLANES  and  would  be  rather  difficult  to  fix.  22  of  the  difficult  errors 
we  re  due  to  the  query  generator,  where  the  heuristics  led  PLANES  to  answer  a  question 
different  fran  vhat  the  user  had  intended.  For  example,  'when  asked  "Now  many  planes 
had  NOR  hours  in  during  the  time  period?",  i*  counted  all  the  planes  that  had  an  NOR 
hours  field  instead  of  counting  only  planes  that  had  greater  than  0  NOR  hours. 
Nonetheless,  we  are  encouraged  by  the  performance  of  PLANES  and  feel  that  with  a 
moderate  amount  of  further  work  it  -will  prove  to  be  a  useful  practical  program. 

7.  Hhgineering  and  cognitive  science 

During  the  7Q's  the  limitations  of  the  engineering  approaches  of  the  60's  became 
evident  and  the  importance  of  developing  a  cognitive  science  has  become  more  generally 
appreciated.  The  reasons  for  this  are  summarised  below. 

(l  )  Even  in  simpler  settings,  language  use  'was  more  complex  and  varied  than  -was 

expected.  For  example,  even  with  a  question  answering  system  we  see  not  .just  requests 

but  examples  of  speech  act3,  declaratives,  reference  to  discourse  entities, 

metaphorical  and  non-literal  usages  of  language,  and  the  need  for  understanding 

* 

metaknowledge. 

(2)  We  see  a  conflict  between  portability  of  natural  language  systems  and  the 
specialization  of  given  systems.  Systems  that  contain  enough  knowledge  to  be  useful 
in  a  limited  domain  often  are  not  very  portable  —  that  is,  great  amounts  of 
information  must  be  added  to  them  in  order  to  be  useful  for  another  domain.  Portable 
systems  require  large  amounts  of  knowledge  to  be  programmed  into  them  to  make  the 
system  useful  in  the  new  icmain.  We  recognize  now  the  need  for  a  science  of  this 
process.  Furthermore,  we  need  to  appreciate  that  as  reported  in  the  -work  on  the 
"mythical  man/ month"  by  Prooks  [1976]  that  if  ■ we  have  a  project  which  requires  a 
thousand  man  months,  we  canot  solve  that  problem  by  putting  a  thousand  men  on  the 
project  for  one  month.  When  group  size  exceeds  five  or  more  people  most  of  the  time 
is  3pent  on  communication  -with  other  group  members  about  what  they're  doing  and  very 
little  time  is  actually  3pent  in  coding  or  developing  systems.  We  need  a  greater 

understanding  of  the  process  of  writing  3uch  systems  in  order  to  be  able  to  rapidly 

produce  larve  3cale  natural  language  understanding  systems. 
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i.'N  The  representations  that  we  have  developed  to  date  are  inadequate.  They  are 
imprecise.  Tor  example,  the  word  "in"  in  the  phrases  "The  crack  in  the  •vail"  and  "The 
man  in  the  rocn"  represent  very  different  kinds  of  relationships  between  the  words  in 
the  sentences.  Mo  adequate  theory  of  representation  of  3uch  words  has  been  developed 
to  date.  Furthermore,  the  representations  that  -we've  dealt  -with  to  date  are  in  many 
•ways  inappropriate.  A  great  deal  of  effort  has  been  concentrated  on  syntactic 
differences  between  sentences.  A  well  known  example  is  the  following:  "I  saw  the  man 
on  the  hill  with  a  telescope."  Earlier  discussions  of  this  sentence  pointed  out  that 
the  phrase  "with  a  telescope"  could  refer  either  to  the  man,  that  is,  the  man  on  the 
hill  could  have  a  telescope,  or  it  could  refer  to  me;  I  could  see  the  man  through  the 
use  of  the  telescope;  and  each  of  these  -would  lead  to  different  parse  trees. 
However,  -we  now  realize  that  the  most  Important  distinctions  here  are  not  captured  by 
the  syntactic  structures, ’ but  are  really  at  the  level  of  inferences,  e.g.  about  where 
I  am  'with  respect  to  the  man  on  the  hill  and  where  the  telescope  is  ’with  respect  to 
both  of  us,  as  illustrated  in  Figure  o- 

Figure  6  illustrates  that  by  adding  other  sentences  to  give  greater  context  the 
sentence  is  no  longer  ambiguous.  The  process  of  both  being  able  to  represent 
appropriately  the  different  meanings  of  this*  sentence  and  being  able  to  find  the 
appropriate  representation  given  to  two  or  more  sentences  in  context  is  3till  a 
outstanding  problem. 

(i'l  Finally,  many  types  of  language  3imply  could  not  be  dealt  -with  at  all.  is 
examples  of  these  types  of  language,  we  can  list  physical  events  and  actions, 
iescriptions  of  scenes,  maps,  paths,  and  instructions;  real  conversations,  arguments, 
debates,  discussions;  language  about  ^notions,  accounts  of  inner  experience: 
meta- iescriptions  and  theories;  situation  assessment  reports;  poetic  language, 
humor,  irony,  lies;  etc. 

3. Current  Topics  in  Natural  Language  Understanding 

At  this  point  I  would  like  to  iiscuss  several  current  topics  and  gv.'J  vw» 
feeling  for  the  kinds  of  research  that  are  being  done  now  -  the  -dr.is  of  vies*",  'ns 
that  are  being  asked.  The  topics  I  would  like  to  loofC  at  incl  ide  soever,  a'to: 
handling  of  novel  language,  especially  metaphor;  applied  natural  language 
understanding  systems:  the  handling  of  natural  language  that  refers  to  conceal 
events;  and  the  inclusion  of  "common  sense"  in  natural  language  mierstanding 
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?. '  Speech  Acts 

Speech  acts  are  per/asive  in  the  behavior  of  users  coward  natural  language 
systems.  As  an  example,  consider  this  invented  but  possible  dialogue: 

User:  Are  there  summaries  for  January? 

System:  Yes. 

User:  Could  I  have  the  January  summaries? 

System:  Yes. 

'Jser:  I  would  like  the  January  summaries. 

System:  I  understand. 

'Jser:  Where  are  the  January  summaries? 

System:  I  don't  'understand  that. 

User:  Can  you  give  me  the  January  summaries? 

System:  Yes,  I  already  told  you  that. 

User:  Would  you  please  give  me  the  summaries  for  January? 


'While  this  discourse  seems  rather  ludicrous  for  a  system  -with  a  wide  range  of 
possible  behavior,  it  corresponds  to  a  common  problem  -with  real  systems,  namely  what 
Kaplan  ^  1 9'73]  has  called  "stonewalling"  behavior.  'One  possible  solution  is  to  view 
each  luery  as  an  instruction  to  select  the  closest  matching  procedure  that  the  system 
has  available  to  it.  Such  a  solution  is  used  in  PLANUS  as  well  as  in  other  current 
generation  systems.  A  second  solution  ^lich  is  only  being  investigated  in  a  basic 
research  context  is  to  model  the  beliefs,  intentions  and  goals  of  the  user.  While 
this  second  solution  i3  much  more  difficult,  it  has  a  great  theoretical  interest  and 
is  probably  the  only  solution  which  ’.will  ultimately  result  in  a  system  that  has  a 
really  satisfactory  understanding  of  the  requests  of  users.  To  give  an  idea  of  how 
difficult  this  second  solution  is,  it  i3  estimated  that  Hhglish  contains  over  1  JOG 
words  each  representing  different  kinds  of  speech  acts.  Por  example,  agree,  request, 
inform,  3tate,  demand,  and  imply  are  all  different  speech  acts. 


Another  complicating  factor  in  speech  act3  i3  that  a  single  given  speech  act  can 
serve  many  purposes.  As  an  example,  the  sentence  "That  cake  that  you  made  looks 
delicious"  might  (l  )  inform  the  hearer  of  one's  opinion  about  the  cake:  (?'  inform 
the  hearer  of  one's  opinion  about  the  oook'3  competence;  praise  the  cook  or.  '  1' 

request  3ome  cake  (or  30m e  more  cake'.  As  another  example,  the  sentence.  "No  planes 
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crashed  in  January,  right?"  both  infoms  the  hearer  of  the  speaker's  belief  and  asks 
for  confirmation  of  the  user's  belief.  In  general,  the  understanding  of  speech  aot3 
is  essential  for  any  system  -which  is  to  understand  stories  that  contain  dialogue,  for 
understanding  conversations,  legal  arguments,  intelligence  reports,  political 
statements,  as  well  as  user  input  to  natural  language  systems,  and  so  on. 

3.2  'lovel  Language 

3ven  though  a  system  may  understand  a  large  number  of  individual  :*ords,  -words  can 
be  used  in  combination  to  form  concepts  that  are  not  easily  understood  as  a  simple 
combination  of  the  sum  of  the  parts.  For  example,  consider  the  phrase  "Water  pump 
pulley  adjustment  screw  threads  damage  report  summary".  A  system  may  be  capable  of 
understanding  each  of  the  individual  -words  and  yet  the  overall  phrase  refers  to  a 
concept  that  is  novrtuere  in  the  dictionary.  Recent  work  by  Finin  [l^SOl  and  others  has 
investigated  how  one  could  develop  productive  rules  for  generating  the  meaning  of  such 
long  noun  phrases.  As  another  example,  consider  the  sentence,  "The  tiger  ran  quickkly 
through  the  jungle."  In  order  to  'understand  the  meaning  of  this  sentence,  one  must 
realise  that  a  tiger  running  quickly  through  the  jungle  -will  run  at  a  different  3peed 
than  a  tiger  running  quickly  across  a  plain.  That  is  to  say,  the  words,  "ran  quickly"  » 

are  relative  to  the  terrain  through  -which  the  tiger  is  running.  We  do  not  have  good 

facilities  for  handling  language  of  this  type.  (By  "handling",  I  mean  being  able  to 
represent  the  different  possible  meanings  differently,  and  being  able  to  decide  -which 
meaning  is  intended  in  a  given  instance.) 

Another  important  category  of  novel  language  includes  metaphor,  simile  and 
analogy.  Hiring  recent  years  -we  have  cane  to  'understand  much  better  ‘now  pervasive 
such  phenomena  are  in  language.  Metaphor  can  be  broadly  divided  into  two  types,  -.which 
I  will  call  "small"  and  "large”.  .As  an  example  of  "snail"  metaphor,  there  is  the 
sentence,  "The  thought  escaped  me  like  a  squirrel  darting  behind  a  tree"  Hr  tony 
In  this  example  note  that  we  have  no  -way  of  talking  about  behavior  of  thought 
ether  than  metaphorically.  Tven  3aying  "the  thought  escaped  me"  is  using  a  metaphor. 
Also  note  that  in  order  to  understand  this  we  have  to  have  some  notion  of  che  behavior 

of  squirrels,  the  physical  meaning  of  the  res4-  of  the  metaphor. 

As  an  example  of  a  large  metaphor,  consider  what  I  call  the  "hydraulic  metaohor 
for  economics."  The  hydraulic  metaphor  can  govern  lame  portions  of  a  text,  oerhaDS  an 
entire  book  on  economics.  We  need  the  hydraulic  mecanhor  *o  understand  terms  like 
pressure,  accumulation,  cash  flow,  cash  reservoirs,  inflationary  pressure,  draining  of 
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resources”,  etc.  As  another  example  of  a  large  metaphor,  consider  the  "conduit 
metaphor”  for  communication,  -which  treats  language  as  though  thoughts  -were  capable  of 
being  put  into  a  bundle  ami  sent  through  a  pipe  to  the  hearer  where  they  are 
unpackaged  and  inspected.  Thus,  we  can  say,  "You  aren't  getting  your  ideas  across  to 
me"  or  "I  gave  her  some  good  ideas".  Other  metaphors  are  possible  for  communication 
including  the  "radio  link"  metaphor  as  in,  "We  were  on  the  same  -wavelength"  or  "I  hear 
you  loud  and  clear".  It  seems  that  some  concepts  can  only  be  treated  metaphorically 
and  have  no  neutral  terms  in  which  to  be  expressed.  .As  an  example,  love  can  be  talked 
about  as  though  it  were  a  team  effort,  a  physical  connection  or  bond,  a  master-slave 
relationship,  a  resonance  between  two  people,  a  journey,  complementary  shapes, 
sharing,  fighting  or  contention,  madness,  etc. 

For  'understanding  novel  language  we  can  identify  two  extreme  approaches.  One  is 
to  have  a  number  of  canned  concepts  coupled  with  weak  matching  rules  to  select  the 
most  appropriate  canned  concept.  This  approach  has  been  used  to  date  for  most  natural 
language  understanding  systems.  In  general,  however,  we  need  other  kinds  of  methods 
for  reasoning  about  and  ultimately  producing  new  concept  representations  given  old 
concept  representations.  Such  methods  have  only  begun  to  be  explored. 

» 

3.3  Plausibility  Judgement  and  Common  Sense 

The  third  current  research  topic  I  would  like  to  discuss  is  the  problem  of 
modeling  common  sense  and  plausibility  judgement  in  a  language  system.  In  general, 
natural  language  processing  systems  have  had  no  connection  with  the  perceptual  'world. 
That  is  to  say,  their  only  channel  to  the  outside  -world  has  been  through  language.  In 
a  strict  sense,  3uch  systems  cannot  be  said  to  know  what  they  are  talking  about,  but 
can  only  know  how  to  talk  about  things.  They  have  no  connection  to  the  perceptual 
■world,  no  more  than  rudimentary  plausibility  judgement,  no  ability  to  handle  language 
about  scenes,  physical  events  and  objects  shapes,  no  good  way  for  handling  metaphors 
that  attempt  to  interpret  the  abstract  world  in  terms  of  the  3ensory-motor  -world,  and 
no  facility  or  even  hope  of  a  facility  for  doing  realistic  reasoning  from  experience 
(except  linguistic  experience). 

Thus,  current  systems  are  unable  to  handle  words  such  as  attract,  repel,  divide, 
separate,  connect,  join,  shatter,  smash,  scratch,  cut,  slice,  crack,  touch,  hit,  lean, 
support,  hang,  bounce,  warp,  -wear,  bend,  tear,  chip,  crease,  etc.  'tote  that  these 
words  are  extremely  important  not  only  in  describing  the  physical  -world  but  also  in 
describing  abstract  worlds,  in  particular  relationships  and  interactions  between 
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people.  Furthermore,  no  current  systems  can  handle  adverbial  modifiers  such  as 
almost,  violently,  gently,  hard,  suddenly,  fast,  slow,  etc. 

Plausibility  judgements  are  important  even  in  the  rather  simple  world  of  question 
answering  systems  for  avoiding  costly  data  base  or  memory  search  in  the  cases  where 

questions  are  asked  that  simply  make  no  sense.  For  example,  if  asked,  "Did  any  plane 

crash  more  than  five  times  last  month?",  a  system  should  not  have  to  go  to  the  data 
base  and  search  it  in  order  to  answer  the  question,  since  crashes  ’usually  happen  to  a 
given  plane  only  once.  It  is  possible  that  a  user  would  expect  a  system  to  interpret 
"any  plane”  as  referring  to  a  class  of  aircraft,  e.g.  A7,  F4,  DC-10,  or  747.  Note 

that  in  order  to  make  this  interpretation  of  this  sentence,  however,  a  system  would 

have  to  understand  that  "any  plane"  could  not  refer  very  meaningfully  to  a  3ingle 
aircraft,  even  though  the  language  of  the  sentence  would  typically  suggest  that  only  a 
single  aircraft  was  intended,  as  in  "Did  any  plane  have  more  than  ten  hours  of 
maintenance  last  month?"  In  this  latter  case  we  would  not  want  to  interpret  "any 
plane"  as  referring  to  DC-10  or  F4  but  rather  as  referring  to  a  specific  individual 
aircraft . 

As  another  example  of  the  need  for  plausibility  judgement  consider  the  following: 
"How  many  propellor  replacements  we*re  marie  for  A4's?"  In  this  case  a  system  that 
looked  for  propeller  replacement  examples  when  A4's  had  no  propellers  would  ’waste 
resources  in  a  serious  manner.  Here  the  problem  is  how  to  represent  the  equipment  or 
nature  of  the  items  in  the  data  base  so  that  the  data  base  search  could  be  avoided. 
As  another  example,  consider  the  following:  "We  were  afraid  the  milk  might  make  the 
baby  sick,  so  we  boiled  it."  Here,  in  order  to  to  realize  that  "it"  must  refer  to  the 
milk  and  not  to  the  baby,  a  system  must  ’understand  the  ordinary  behavior  of  people,  in 
particular  that  people  might  boil  milk  but  would  rarely  boil  babies. 

We  have  recently  made  progress  in  dealing  with  simple  aspects  of  space.  Since 
space  is  a  topic  that  can  be  shared  by  many  possible  natural  language  system 
applications,  many  possible  domains,  we  feel  it  is  useful  to  consider  it  in  the 
abstract. 

Programs  written  by  Lois  3oggess  [l973][Wsltz  and  3oggess  1979]  can  deal  ’with  the 
following  types  of  sequences  of  input:  The  goldfish  is  in  a  goldfish  bowl;  the 
goldfish  bowl  i3  on  a  shelf;  the  shelf  is  on  the  desk;  the  desk  is  in  a  room.  If 
given  a  question,  "Is  the  goldfish  in  the  room?"  the  system  can  answer  "yes"  by 
referring  to  a  representation  that  it  builds  as  3hown  in  Figure  7. 


writing  implement,  'Aich  is  rather  fixed  in  size  and  smaller  than  most  boxes.  The 
pen,  in  this  case,  more  likely  refers  to  a  play  pen  or  a  stock  pen.  Using  a  program 
similar  to  Boggess's,  mentioned  above,  we  can  note  that  boxes  have  a  size  range  from 
oerhaps  two  inches  on  a  side  to  five  feet  on  a  3ide,  that  they  are  hollow  containers, 
and  that  pens  —  that  is,  writing  implements  —  are  on  the  order  of  3ix  inches  long 
and  a  half- inch  in  diameter  and  relatively  fixed  in  size  and  not  hollow  containers; 
that  pens  —  play  pens  —  are  roughly  four  feet  on  a  side  and  hollow  containers. 
Stock  pens  are  even  larger.  When  a  system  tries  to  construct  a  special  representation 
of  a  box  within  a  pen  (writing  implement)  the  system  is  simply  incapable  of  creating 
such  a  relationship.  On  the  other  hand,  in  the  case  of  a  box  'within  a  playpen  or 
stock  pen,  the  representation  could  be  easily  constructed  and  the  system  can  thus 
plausibly  judge  that  "pen"  in  this  case  must  refer  to  stock  pen  or  playpen. 

3.4  Progress 

In  this  section  I  would  like  to  point  out,  for  the  interested  reader,  a  number  of 
recent  projects  that  premise  to  solve  some  of  the  outstanding  problems  of  natural 
language  understanding. 

The  first  topic  of  interest  is  knowledge  representation,  k  number  of  pieces  of 
work  in  recent  years  have  led  to  considerable  progress  in  the  knowledge  representation 
area.  The  -CL -ONE  system  of  Brachman  f 1 979]  is  general  enough  to  represent  grammar 
rules,  semantic  interpretation  rules,  speech  act  rules,  as  well  as  object  and  event 
taxonomies.  XL -ONE  is  a  language  in  which  many  different  kinds  of  knowledge  can  be 
expressed  in  a  uniform  manner,  and  shared  between  different  components  of  a  full 
natural  language  understanding  system.  XL-ONE  was  inspired  by  the  "procedural 
semantics"  ideas  of  Wood3  [l979]. 

Other  work  on  representing  mechanisms  and  the  geometry  of  objects  has  been  done 
by  Rieger  [l975],  Hayes  [l973],  Forbus  ij979],  deKleer  [l977,  1979],  and  Waltz  [1970]. 
Work  in  representing  physical  scenes  and  events  has  been  done  by  3oggess  [l  9^3 ] ,  Waltz 
:J930a,b],  Herskovitz  [l980],  and  Johnson- Laird  1 979 ] .  Representation  of  large  scale 
space  maps  has  been  explored  by  Xuipers  [ 1 9^3]  and  McDermott  [ 1 930 ] . 

Other  work  in  knowledge  representation  has  attempted  to  deal  with  problems  of 
inconsistent  kiowledge  —  that  is,  the  problem  of  how  to  add  new  information  to  a 
3y3tem  which  may  conflict  with  information  presently  in  a  system.  Work  on 
"non-monotonic  logic"  to  deal  'with  3uch  conflicts  has  been  done  by  Doyle  [l973,  1930', 
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McDermott  [1979],  Weyrauch  J979]  and  others. 

Some  other  examples  of  general  progress  are  in  the  area  of  summarizing  and 
translating  newspaper  articles,  modeling  emotional  conflicts  and  reactions,  modeling 
argumentation,  writing  psychologically  realistic  parsers,  making  simple  natural 
language  front  ends  commercially  available,  and  understanding  the  meanings  of  phrases. 
Recently  there  has  been  a  great  deal  of  interest  in  the  generation  of  language  and 
also  renewed  interest  in  the  area  of  speech  ’understanding.  Rirthermore ,  many  natural 
language  parsers  are  approaching  closure,  that  is,  being  able  to  handle  all  naturally 
occurring  grammatical  types  of  sentences  (see,  for  example,  [Bobrow  I960]). 

9.  Summary 

I  have  tried  to  show  in  this  paper  how  ideas  have  progressed  from  the  point  -where 
we  first  understood  that  computers  could  be  used  for  processing  text  and  general 
concepts  as  well  as  numbers  to  the  point  where  simple  mechanisms  for  dealing  -with 
language  were  tried  but  discarded  for  machine  translation,  through  an  era  of  attempts 
to  handle  natural  language  processing  through  any  means  available,  through  the 
engineering  of  systems  that  deal  with  simplified  natural  language  in  narrow  domains. 
We  are  now  at  a  phase  where  we  have  begun  to  realize  that  in  order  to  deal  -with 
natural  language,  we  have  to  understand  better  how  it  is  that  people  process  language, 
so  our  emphasis  has  shifted  from  engineering  to  cognitive  science.  If  we  are  to  have 
natural  language  understanding  systems  that  are  truly  satisfactory,  it  must  be  the 
case  that  natural  language  systems  make  appropriate  inferences  about  the  natural 
language  of  people.  It  must  also  be  the  case  that  if  a  computer  system  presents 
3ngli3h  output  to'  users,  the  user  is  justified  in  making  the  inferences  one  -would 
ordinarily  make,  given  that  language.  In  order  to  be  able  to  meet  these  two  criteria, 
natural  language  systems  must  not  simply  understand  the  shallow  surface  meaning  of 
language,  but  must  also  be  able  to  'understand  the  deeper  implications  and  inferences 
that  a  user  is  likely  to  intend  and  likely  to  take  frcm  language.  In  order  to  do 
this,  the  systems  must  be  capable  of  understanding  user  goals,  intents,  and 
strategies,  as  well  as  multiple  purposes  served  by  any  given  piece  of  language. 

■Secondly,  we  have  come  to  ’understand  much  more  clearly  that  if  -.we  are  to  ere r 
build  natural  language  systems  with  both  depth  and  breadth,  we  must  come  to  grips  -with 
either  the  problem  of  learning  from  experience,  or  the  problem  of  designing  and 
building  software  systems  of  a  3cope  and  subtlety  beyond  anything  yet  accomplished. 
In  either  case,  we  lack  the  knowledge  of  how  to  proceed.  It  seems  arrogant  to  assume 


that  we  could  program  a  natural  language  system  to  reach  adult  competence  in  language 
in  anything  less  than  the  twenty  years  required  by  humans,  and,  as  argued  by  Brooics, 
we  cannot  simply  accomplish  this  task  by  putting  more  and  more  people  on  the  same 
project.  Not  only  is  it  difficult  to  coerce  such  people  to  -work  effectively  as  a  team 
because  of  the  sheer  amount  of  intercanmunication  necessary;  such  an  approach  to 
•writing  large  natural  language  processors  also  requires  that  ore  -understand  all  the 
knowledge  representation  schemes  ahead  of  time  so  that  all  team  members  can  generate 
code  portions  that  will  work  properly  together.  At  the  present  such  a  massive 
approach  to  a  natural  language  understanding  system  is  simply  not  feasible,  and  there 
seems  to  be  no  prospect  for  anything  other  than  narrow  dcmain  natural  language  systems 
for  the  foreseable  future.  Areas  that  still  need  a  great  deal  of  work  include 
representation  of  space,  time,  events,  human  behavior,  emotions,  physical  mechanisms, 
and  many  processes  associated  with  metaphor.  Furthermore,  we  must  face  the  problems 
associated  with  learning  from  experience.  Sven  if  we  are  able  to  program  a  system 
ortiich  has  adult  competence  in  language,  such  a  system,  if  it  is  to  display  language 
processing  behavior  like  an  adult,  must  also  be  capable  of  learning  and  dealing  -with 
new  concepts  that  are  taught  to  it  by  a  user  or  through  experience.  We  as  yet  have 
very  few  ideas  on  how  to  deal  with  such  phenomena. 

Finally,  at  this  point  in  history  there  are  many  opportunities.  We  have  some 
natural  language  systems  which  are  already  useful  and  a  number  of  others  which  should 
be  usefully  applied  within  the  near  future.  We  also  have  seen  continued  dramatic 

improvements  and  increases  in  the  power  of  available  hardware  and  software.  Fach 
advance  brings  real  time  natural  language  processing  closer  and  closer.  Natural 
language  systems  have  for  their  entire  history  pushed  the  limits  of  available 

computation,  and  increases  in  the  computational  power  available  to  users  will  clearly 
aid  in  the  solution  of  natural  language  processing  problems,  'tost  computers  have  not 
been  designed  to  work  well  with  natural  language  processing  systems.  lorn outers  have 
been  tuned  primarily  for  numerical  problems.  With  the  -wide  availability  of  VLSI 

technology  it  will  be  possible  for  natural  language  processing  researchers  to  specify 
and  obtain  computers  which  have  architectures  appropriate  to  the  natural  language 

processing  tasks.  Some  natural  candidates  for  improvements  in  this  area  include  true 
associative  memories,  memories  -with  highly  distributed  processing,  separate  processors 
for  syntactic,  semantic,  pragmatic,  and  processing  phases  of  language,  hard-ware  for 
analyzing  speech,  and  so  on.  Already  work  i3  underway  in  these  areas. 
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